CI: Add Alpine Linux 3.23 runner to the pipeline#18087
CI: Add Alpine Linux 3.23 runner to the pipeline#18087tonyhutter merged 1 commit intoopenzfs:masterfrom
Conversation
You could add a textbox to the diff --git a/.github/workflows/zfs-qemu.yml b/.github/workflows/zfs-qemu.yml
index db4fe3bdd..21ffe9aae 100644
--- a/.github/workflows/zfs-qemu.yml
+++ b/.github/workflows/zfs-qemu.yml
@@ -10,6 +10,12 @@ on:
required: false
default: ""
description: "(optional) Experimental kernel version to install on Fedora (like '6.14' or '6.13.3-0.rc3')"
+ specific_os:
+ type: string
+ required: false
+ default: ""
+ description: "(optional) Only run on this specific OS (like 'fedora42' or 'alpine3-23')"
+
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
@@ -63,6 +69,9 @@ jobs:
os_json=$(echo ${os_selection} | jq -c)
fi
+ if {{ github.event.inputs.specific_os != '' }}; then
+ os_json='["'{{ github.event.inputs.specific_os }}'"]'
+ fi
echo "os=$os_json" | tee -a $GITHUB_OUTPUT
echo "ci_type=$ci_type" | tee -a $GITHUB_OUTPUT |
Pull request: openzfs#18087 Signed-off-by: Alexander Moch <mail@alexmoch.com>
Add an Alpine Linux 3.23 runner to the CI chain to run OpenZFS builds and tests against musl libc. Currently, zfs_send_sparse is killed after 10 minutes on Alpine, causing cascading EBUSY failures in the test suite. With zfs_send_sparse disabled, the ZFS test suite reaches a pass rate of 94.62%. This commit introduces the required Alpine-specific setup and a small set of shell and cloud-init compatibility fixes that also apply to existing Linux runners. The Alpine runner is not enabled by default and is not executed for new pull requests. Sponsored-by: ERNW Research GmbH - https://ernw-research.de/ Signed-off-by: Alexander Moch <amoch@ernw.de>
|
Thanks for the input and merging #18085 so quickly; with that merged, the modules now build successfully on the Alpine runner. I’ve rebased this pull request onto the current |
behlendorf
left a comment
There was a problem hiding this comment.
Thanks for the detailed comment explaining the required changes. That definitely helped head off some questions I would have otherwise had! This LGTM. Once we're confident this builder will regularly pass the test suite we can also consider enabling it by default. Better to catch any build issues in the original PRs for new changes.
|
This pull request only has a single commit - nothing to squash. There's probably some confusion because I have a separate development branch ( |
I thought that getting this into |
Add an Alpine Linux 3.23 runner to the CI chain to run OpenZFS builds and tests against musl libc. Currently, zfs_send_sparse is killed after 10 minutes on Alpine, causing cascading EBUSY failures in the test suite. With zfs_send_sparse disabled, the ZFS test suite reaches a pass rate of 94.62%. This commit introduces the required Alpine-specific setup and a small set of shell and cloud-init compatibility fixes that also apply to existing Linux runners. The Alpine runner is not enabled by default and is not executed for new pull requests. Sponsored-by: ERNW Research GmbH - https://ernw-research.de/ Signed-off-by: Alexander Moch <amoch@ernw.de> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Add an Alpine Linux 3.23 runner to the CI chain to run OpenZFS builds and tests against musl libc. Currently, zfs_send_sparse is killed after 10 minutes on Alpine, causing cascading EBUSY failures in the test suite. With zfs_send_sparse disabled, the ZFS test suite reaches a pass rate of 94.62%. This commit introduces the required Alpine-specific setup and a small set of shell and cloud-init compatibility fixes that also apply to existing Linux runners. The Alpine runner is not enabled by default and is not executed for new pull requests. Sponsored-by: ERNW Research GmbH - https://ernw-research.de/ Signed-off-by: Alexander Moch <amoch@ernw.de> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Add an Alpine Linux 3.23 runner to the CI chain to run OpenZFS builds and tests against musl libc. Currently, zfs_send_sparse is killed after 10 minutes on Alpine, causing cascading EBUSY failures in the test suite. With zfs_send_sparse disabled, the ZFS test suite reaches a pass rate of 94.62%. This commit introduces the required Alpine-specific setup and a small set of shell and cloud-init compatibility fixes that also apply to existing Linux runners. The Alpine runner is not enabled by default and is not executed for new pull requests. Sponsored-by: ERNW Research GmbH - https://ernw-research.de/ Signed-off-by: Alexander Moch <amoch@ernw.de> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Add an Alpine Linux 3.23 runner to the CI chain to run OpenZFS builds and tests against musl libc. Currently, zfs_send_sparse is killed after 10 minutes on Alpine, causing cascading EBUSY failures in the test suite. With zfs_send_sparse disabled, the ZFS test suite reaches a pass rate of 94.62%. This commit introduces the required Alpine-specific setup and a small set of shell and cloud-init compatibility fixes that also apply to existing Linux runners. The Alpine runner is not enabled by default and is not executed for new pull requests. Sponsored-by: ERNW Research GmbH - https://ernw-research.de/ Signed-off-by: Alexander Moch <amoch@ernw.de> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Add an Alpine Linux 3.23 runner to the CI chain to run OpenZFS builds and tests against musl libc. Currently, zfs_send_sparse is killed after 10 minutes on Alpine, causing cascading EBUSY failures in the test suite. With zfs_send_sparse disabled, the ZFS test suite reaches a pass rate of 94.62%. This commit introduces the required Alpine-specific setup and a small set of shell and cloud-init compatibility fixes that also apply to existing Linux runners. The Alpine runner is not enabled by default and is not executed for new pull requests. Sponsored-by: ERNW Research GmbH - https://ernw-research.de/ Signed-off-by: Alexander Moch <amoch@ernw.de> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
Add an Alpine Linux 3.23 runner to the CI chain to run OpenZFS builds and tests against musl libc. Currently, zfs_send_sparse is killed after 10 minutes on Alpine, causing cascading EBUSY failures in the test suite. With zfs_send_sparse disabled, the ZFS test suite reaches a pass rate of 94.62%. This commit introduces the required Alpine-specific setup and a small set of shell and cloud-init compatibility fixes that also apply to existing Linux runners. The Alpine runner is not enabled by default and is not executed for new pull requests. Sponsored-by: ERNW Research GmbH - https://ernw-research.de/ Signed-off-by: Alexander Moch <amoch@ernw.de> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>

Add an Alpine Linux 3.23 runner to the CI chain to run OpenZFS builds and tests against musl libc.
Currently, zfs_send_sparse is killed after ~10 minutes on Alpine, causing cascading EBUSY failures in the test suite. With zfs_send_sparse disabled, the ZFS test suite reaches a pass rate of 94.62%.
This commit introduces the required Alpine-specific setup and a small set of shell and cloud-init compatibility fixes that also apply to existing Linux runners.
The Alpine runner is not enabled by default and is not executed for new pull requests.
Motivation and context
The CI chain currently does not cover musl-based Linux systems. Bugs on these systems are therefore often reported after new releases. This pull request is meant to catch such issues earlier, before they hit downstream.
Disclaimer: I am co-maintaining the
sys-fs/zfspackage for Gentoo and, even though musl-based profiles are marked as experimental, I care about this use case.Incidentally, work on this pull request already resulted in the following bug fix:
PATH_MAXstack allocation inztest_get_zdb_bin()#18085So, I assume the Alpine CI does what it is supposed to do. 😉
Description
Parts of this commit are opinionated. I try to pick the simplest solution, keep the diff small, and avoid duplicating code where possible. If a maintainer prefers a different approach, I am very open to adjusting this.
The runner passes 94.62% of the ZFS test suite with
zfs_send_sparsedisabled. This is a solid starting point, but not sufficient for a green CI run.zfs_send_sparsecurrently times out and causesEBUSYfailures that cascade through the test suite. There are also unresolved issues aroundatimeandhostid.While a fully green CI would be desirable prior to enabling the runner, this pull request deliberately limits its scope to adding Alpine support without enabling it for new pull requests. As such, the primary risk of merging lies in a small set of compatibility changes that affect existing runners.
94.62% is a good baseline to continue improving the runner and eventually enable it once it reliably passes the full ZFS test suite.
What has been changed?
It is important to distinguish between changes that affect existing runners and those that are specific to the new Alpine runner. This commit does not touch
zfs-qemu.yml. Therefore, the Alpine runner will not be executed when merging this pull request.Changes affecting current runners
Some compatibility changes required for Alpine also affect the other runners in the CI chain. Most notably, the cloud-init configuration was adjusted. Aside from minor formatting changes, the relevant points are:
$BASHwith/bin/bash.$BASHis not set on Alpine. Bash is located at/bin/bashon all Linux distributions in the CI chain.rootto executesudowithout a password.lock_passwd: falseandpasswd: '*'.openssh-server-pamand enabling PAM insshd_config. Given that this runs in ephemeral CI VMs, this approach keeps the configuration simple and avoids per-distro branching.bashandsudopackages.The fourth point relies on
aptfor Debian-based systems, socloud-init status --waitis needed to avoid hanging due to anaptlock. Adding this globally caused AlmaLinux 8 to hang, so it is limited to Debian and Ubuntu. Overall, waiting for cloud-init improves robustness.In some scripts the Alpine runner initially complained about
bashnot being found, likely due to environment differences. These calls were changed to POSIX-compatiblesh, as they are only used for output redirection withsudo.Alpine mounts loop devices as
/dev/loop/Xinstead of/dev/loopX, which required a small regular expression adjustment. Finally, a syntax fix inlibtest.shlibprevents cleanup failures.The new Alpine Linux runner
Alpine has a few peculiarities. One of them is
kshnot being available in the repositories, so it is built from source, which takes a few minutes. Logging into the VMs also requires manually disabling cloud-init services during setup. The system is switched frommdevtoeudev, and/etc/zfs/zpool.cacheis created manually.The remaining changes are what would be expected for a new Linux runner.
Possible future directions
This section lists ideas that could be explored in the future, but are likely out of scope for this pull request.
How has this been tested?
Several test runs using GitHub Actions and the ZFS test suite. Alpine currently reaches a 94.62% pass rate.
All other distributions pass the CI pipeline after these changes.
Types of changes
Checklist
Signed-off-by.